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Abstract — We consider a distributed source coding problem 
of L correlated Gaussian observations Yi,i = 1,2, L. We as- 
sume that the random vector = '(Yi, F2, ■ • , Vl) is an obser- 
vation of the Gaussian random vector X'^ {Xi ,X2, - ■ ■ , Xk), 
having the form = AX'^ + A^^ , where Aisa Lx K matrix 
and 7V^ — ^{Ni,N2,---, Nl ) is a vector of L independent Gaus- 
sian random variables also independent of X^. The estimation 
error on is measured by the distortion covariance matrix. 
The rate distortion region is defined by a set of all rate vectors 
for which the estimation error is upper bounded by an arbitrary 
prescribed covariance matrix in the meaning of positive semi 
definite. In this paper we derive explicit outer and inner bounds 
of the rate distortion region. This result provides a useful tool 
to study the direct and indirect source coding problems on this 
Gaussian distributed source coding system, which remain open 
in general. 

Index Terms — Multiterminal source coding, rate-distortion re- 
gion, CEO problem. 



I. Introduction 

Distributed source coding of correlated information sources 
are a form of communication system which is significant from 
both theoretical and practical points of view in multi-user 
source networks. The first fundamental theory in those coding 
systems was established by Slepian and Wolf 0]. They con- 
sidered a distributed source coding system of two correlated 
information sources. Those two sources are separately encoded 
and sent to a single destination, where the decoder reconstruct 
the original sources. 

In the above distributed source coding systems we can con- 
sider the case where the source outputs should be reconstructed 
with average distortions smaller than prescribed levels. Such 
a situation suggests the multiterminal rate distortion theory. 

The rate distortion theory for the above distributed source 
coding system formulated by Slepian and Wolf has been 
studied by [|2]-I9J. Wagner et al. liOl gave a complete solution 
to this problem in the case of Gaussian information sources 
and quadratic distortion by proving that sum rate part of the 
inner bound of Berger |4|| and Tung [|5] is tight. Wang et al. 
ifTTl gave a new alternative proof. 

As a practical situation of distributed source coding systems, 
we can consider a case where the distributed encoders can 
not directly access to the source outputs but can access to 
their noisy observations. This situation was first studied by 
Yamamoto and Ito 1121 . They call the investigated coding 
system the communication system with a remote source. 
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Subsequently, a similar distributed source coding system was 
studied by Flynn and R. M. Gray |fT3]| . 

In this paper we consider a distributed source coding 
problem of L correlated Gaussian sources Yi,i = 1,2, - ■ ■ ,L 
which are noisy observations of Xi,i = 1,2, ■ ■ ■ , K. We 
assume that = '(Yi,y2, ■ ■ ■ is an observation of 
the source vector X'^ = *^{Xi, X2, ■ ■ ■ , Xk), having the 
form = AX^ + , where A is sl L x K matrix 
and = ^{Ni,N2, ■ ■ ■ ,Nl) is a vector of L independent 
Gaussian random variables also independent of X^ . 

We consider two distortion criterions based on the covari- 
ance matrix of the estimation error on Y^ . One is the criterion 
called the vector distortion criterion distortion region where 
each of the the diagonal elements of the covariance matrix 
is upper bounded by a prescribed level. The other is the 
criterion called the sum distortion criterion where the trace 
of the covariance matrix is upper bounded by a prescribed 
level. For each of the above two distirion criterions we derive 
explicit inner and outer bounds of the rate distiron region. We 
also derive an explicit matching condition in the case of the 
sum distortion criterion. 

When K = 1, the source coding system becomes that of the 
quadratic Gaussian CEO problem investigated by ifTTl . 1(141 - 
[|T6| . The system in the case of K = L and sum distortion 
criterion was studied by Pandya et al. ||T7| . They derived 
lower and upper bounds of the minimum sum rate in the 
rate distortion region. Several partial solutions in the case of 
K = L, A = II and sum distortion criterion are obtained by 
lfT8l - ll22l . The case of K = L, A = II and vector distortion 
criterion is studied by ||20l . 

The remote source coding problem treated in this paper 
is also referred to as the indirect distributed source coding 
problem. On the other hand, the multiterminal rate distortion 
problem in the frame work of distributed source coding is 
called the direct distributed source coding problem. As shown 
in the paper of Wagner et al. ifTOl and in the recent work by 
Wang et al. IfTTl . we have a strong connection between the 
direct and indirect distributed source coding problems. 

In this paper we also consider the multiterminal rate dis- 
tortion problem, i.e., the direct distributed source coding 
problem for the Gaussian information source specified with 
Y^ = X^ + N^, which corresponds to the case of K = L 
and A = II. We shall derive a result which implies a strong 
connection between the remote source coding problem and the 
multiterminal rate distortion problem. This result states that all 
results on the rate distortion region of the remote source coding 
problem can be converted into those on the rate distortion 
region of the multiterminal source coding problem. Using this 
result, we drive several new partial solutions to the Gaussian 
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Fig. 1. Distributed source coding system for L correlated Gaussian 
observations 



multiterminal rate distortion problem. 

II. Problem Statement and Previous Results 
A. Formal Statement of Problem 

In this subsection we present a formal statement of problem. 
Throughout this paper all logarithms are taken to the base 
natural. Let Xi^i = 1, 2, • • • , X be correlated zero mean 
Gaussian random variable. For each i = l,2,- -,ii', Xi 
takes values in the real line Xi. We write a k dimensional 
random vector as X^ = ^{Xi,X2, ■■■,Xk)- We denote 
the covariance matrix of X^ by Let = *(li,l2, 

• • • , Yl) be an observation of the source vector X^, having 
the form Y^ = AX^ + N^, where A is a i x X matrix and 
iV^ = *(iVi, N2, - ■ ■ , Nl) is a vector of L independent zero 
mean Gaussian random variables also independent of X^^ . 
For i = 1,2, ■ ■ ■ , L, aj^, stands for the variance of Ni. Let 
{{Xi{t), X2{t), ■ ■ ■ be a stationary memoryless 

multiple Gaussian source. For each t = 1,2,- ••, X^^{t) ~ 
*{Xi{t), X2{t), ■ ■■ , Xk{t)) has the same distribution as X'^. 
A random vector consisting of n independent copies of the 
random variable Xi is denoted by 

^ (X,(l),X,(2),..-,X,(n)). 



X, 

For each t = 1, 2, • • •, Yi{t),i = 1,2, - ■ ■ ,L is a vector of L 
correlated observations of X'^{t), having the form Y^(t) ~ 
AX^{t)+N^{t), where N^{t),t = 1,2, are independent 
identically distributed (i.i.d.) Gaussian random vector having 
the same distribution as N^. We have no assumption on the 
number of observations L, which may he L > K 01 L < K. 

The distributed source coding system for L correlated 
Gaussian observations treated in this paper is shown in Fig. 
L In this coding system the distributed encoder functions 
ipi,i = 1,2, - ■ ■ ,L are defined by 
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M, = {l,2,---,M,} . 
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For each i = 1, 2, • • • , L, set ni = ^ log Mi , which stands 
for the transmission rate of the encoder function 1^9'"'. The 
joint decoder function -0'^"^ = ("0^"'', V'2"^ ''''V'if^) is 
defined by 
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where Xi is the real line in which a reconstructed random 
variable of Xi takes values. For ~ (Xi, X2, ■ ■ ■ , X^), 
set 
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da — E| — Xi 
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E(X, - X„ X, -X,),l<i^j <K, 

where | |a| | stands for the Euclid norm of n dimensional vector 
a and {a, b) stands for the inner product between a and b. Let 
^x^'-x* ^ covariance matrix with dij in its entry. 
Let Ed be a given L x L covariance matrix which serves as a 
distortion criterion. We call this matrix a distortion matrix. 

For a given distortion matrix S^, the rate vector {Ri, 
-R2r-',-RL) is 'Sid-admissible if there exists a sequence 
V'("^)}5f=i such that 
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lim sup i? -"^ < Ri, fori- 

n— ^cxD 

lim sup ■k'^x'^-x"' - ■ 



1,2,- 



where Ai ^ A2 means that A2 — Ai is positive semi- 
definite matrix. Let TZLi^dl^x'^Y^) denote the set of all S^- 
admissible rate vectors. We often have a particular interest in 
the minimum sum rate part of the rate distortion region. To 
examine this quantity, we set 
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We consider two types of distortion criterion. For each distor- 
tion criterion we define the determination problem of the rate 
distortion region. 

Problem 1. Vector Distortion Criterion: Fix K x K 
invertible matrix F and positive vector = {Di , D2, - ■ ■ 
, Dk)- For given F and , the rate vector (i?i, R2, - ■ ■ , Rl) 
is {T, D^)-admissible if there exists a sequence {((^^"'', 

such that 

limsupi?*") < R„ for i 

n— >cjo 

lim sup 



1,2, 



r -^E, 



< A , for i = 1,2, 



where [C]ij stands for the entry of the matrix C. 

Let 7^L(^,D^|SxKyL) denote the set of all (r,D^')- 
admissible rate vectors. When T is equal to the K x K 
identity matrix Ik, we omit F in TZl{T, DlY^xKyi-) to 
simply write TZl{D\T,x^y^)- Similar notations are used for 
other sets or quantities. To examine the sum rate part of 
7^L(^,i:)^'|ExK■yL), define 
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x^yi^ ) = nun 
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Problem 2. Sum Distortion Criterion: Fix K x K positive 
definite invertible matrix F and positive D. For given F and D, 
the rate vector i?2, ■ • • , Rl) is (F, D)-admissible if there 
exists a sequence {(tp^^\ ip'"^\ • • • , ^^^\ such that 

linisupi?^") < i?„ for i l,2,---,i, 



and set 



lim sup tr 



< D. 



To examine the sum rate part of TZl{T, DIT^xi^y^)^ define 
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Rsnm,L(r,D\^x'<Y'-j= mm 

67?,t(r,_D|E^Kvi 



Let Sk{D^) be a set of all K x K covariance matrices whose 
{i, i) entry do not exceed Di for i — 1,2, ■ ■ ■ , K. Then we 
have 

7^i(F,i^^|S;fKy.) = U nLi^d\^x'<Y^),a) 

7^t(F,i5|ExKyL) = y 7^L(Erf|SxKyO■ (2) 

tr[rSd«T]<D 

Furthermore, we have 

^xkyl) ■ (3) 

In this paper we establish explicit inner and outer bounds of 
7?,L(S(i|SxA-yi,). Using the above bounds and equations ([U 
and we give new outer bounds of TZl{T, D\T^x'<yl) and 

nL{T,D^\j:x'<Y^)- 



B. Inner Bounds and Previous Results 

In this subsection we present inner bounds of TlLC^d 
nL{T,D^ ISx^yO, and 7^L(^,D \^x'<y^)- 
Those inner bounds can be obtained by a standard technique 
developed in the field of multiterminal source coding. 

Set A = {1,2, - ■ ■ ,L}. For i G A, let Ui be a random 
variable taking values in the real line Ui. For any subset 
S" C A, we introduce the notation Us = {Ui)i^s- In particular 

C/a = C/^ = (C/i, U2, ■■■,Ul). Define 

g{T,d) = {U^ : is a Gaussian 

random vector that satisfies 

for any S* C A and 

for some linear mapping 



A 



conv {i?^ : There exists a random vector 
e ^(Sd) such that 

Ei?^ >/(C/s;i^s|C/sO 

for any S <Z A . } , 
where conv{A} stands for the convex hull of the set A. Set 

7^f)(F,i^^|E;fK, 



A 



Define 



Cyt J 

conv< y 7^L(Ed|SxK-yt) 
rSd'rG5K(D^) 

7^f)(F,I?|E;fKy.) 

= COnV<| y 7^L(Sd|SxA'y^.) 

[tr[rEd'r]<D 



and set 



• • ■ , [^^x'<\yl^^ll) ■ 

We can show that 7?,^"' (SdlSx^y^ ), 7e^"'(F, ZJ^lEx^y^-), 
and T^^'"' (F, DjS^Kyi, ) satisfy the following property. 
Property 1: 

a) The set 7?.^"' (S^lSx^y^ ) is not void if and only if >- 

'^Xi^\Y^- 

b) The set 7^^'"^(F, D^^lS^A-yi.) is not void if and only if 

c) The set "^^'"^ (F, Z?|I]x/<yi ) is not void if and only if 

D > tT[TT,xKiYL*T]. 

On inner bounds of 7?,L(S(i|Ej(-Kyi), TZl{T, D^l'Ex'^Y'- 
), and 7^i(F, we have the following result. 

Theorem 1 (Berger and Tung jj^): For any >- 
Sx^'|y^' we have 

T^L^H'^dl'^X'^Y^) ^hi^dl^Xi^Y^) ■ 

For any F and any D^'^ > d^'(FI]xic|yi.*F), we have 

7^f )(F,i^^'|I];,Ky.) C TZl{T,D''\^x^y-) ■ 
For any F and any D > tr[rSjf R'|yi,*F], we have 

7t^"^(F,Z?|Sx-y.) C7eL(F,Z?|Sx-y.). 

The above three inner bounds can be regarded as variants 
of the inner bound which is well known as that of Berger ||4| 
and Tung 15]. 

When K = 1 and Lxl column vector A has the form A = 
• ■ • 1], the system considered here becomes the quadratic 
Gaussian CEO problem. This problem was first posed and 
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investigated by Viswanathan and Berger lfT4l . They further b) Suppose that G ^^(Sd)- If 



A 



assumed Sjv^ = II- Set cr^ = and 

Rsum{D\a\,a'^) = liminf i?sum.L(£'|Sxyi.) ■ 

L— >CXD 

Viswanathan and Berger lfT4l studied an asymptotic form of 
|fT^,fT2) for small D. Subsequently, Oohama ||T5] 
determined an exact form of Rsun\{D\a\ , a^) . The region 
TI.l{D\T,xy'') was determined by Oohama lfT6l . 

In the case where K = L and F = A = /l, Oohama 
EU-lEol derived inner and outer bounds of 7?.i(Z)|I]xiyt ). 
Oohama |[T9l also derived explicit sufficient conditions for 
inner and outer bounds to match and found examples of infor- 
mation sources for which rate distortion region are explicitly 
determined. In 1201 . Oohama derived explicit outer bounds of 

7^L(Sd iSx^Y^.), 7^L(-D^ ISx^yi.), and TlLiD iSx^-y^)- 
Recently, Wagner et al. ifTol have determined TZ2{D^\ 

Sx^ys). Their result is as follows. 

Theorem 2 (Wagner et al. UUj): For any D"^ > (P{[T,xk\ 

yt]), we have 

Their method for the proof depends heavily on the specific 
property of L = 2. It is hard to generalize it to the case of 
i > 3. 

III. Main Results 

A. Inner and Outer Bounds of the Rate Distortion Region 

In this subsection we state our result on the characterizations 
of 7^L(Ed \T.xkyl), TlLiT.D^ \Exkyl), and nL{T,D 
|ExKyi). To describe those results we define several func- 
tions and sets. For > 0, i G A, let Ni{ri), i G A be 
L independent Gaussian random variables with mean and 
variance ct^./(1 — e"^'"'). Let S^i^^i-) be a covariance matrix 
for the random vector N^{r'"). Fix nonnegative vector r^. For 
9 > Q and for S* C A, define 



y-l A 1 



rs=0 



,2n 



^XK 



where S'^ 



■^XK 



A 



A 



A — S and log x ~ maxjlogx, 0} . Set 

1 



AU^d) = > 



^XK 



'AT. 



A 



We can show that for 5 C A, J_s{\'^d\, rs\rs<^) and 
Js{rs\rs^) satisfy the following two properties. 
Property 2: 

a) If G Aii^d), then for any S C A, 

lsi\^d\,rs\rs-) < Jsirslrs") ■ 



ALi^d), then 



^ still belongs to 



Jsi\^d\,rs\rs<=)l^^o = Js{rs\rs<')\rs=o 
= 0. 

Property 3: Fix G ALi^d)- For 5" C A, set 

fs = fsirslrs") = J_si\^d\,rs\rs'=) . 

By definition, it is obvious that fs,S C A are nonnegative. 
We can show that / = {fs}scA satisfies the followings: 

a) h = 0. 

b) fA< Jb for AC BC A. 

C) fA + fs < fAnB + JauB ■ 

In general (A, /) is called a co-polymatroid if the nonnegative 
function p on 2"^ satisfies the above three properties. Similarly, 
we set 



fs = fsirslrs") = Jsirslrs") , / = 



SCA 



Then (A, /) also has the same three properties as those of 
(A, /) and becomes a co-polymatroid. 
To describe our result on 7?.L(Sd|ExA'yi.), set 



n 



(out) , 



,r^\^x'<Y^) 



A 



{R^ : ^i?. > J5(e,rs|rso) 
for any S" C A . } , 



7^ 



(out) 



= {R^ : ^i?.> J5(rs|rsc) 

for any S" C A . } , 

'^^"''(SdlSxifyi.) 



i^conv<| U nT\r^\llx^y.) 

'■^G.Ai,(Sd) 

We can show that 7^^'"'(S<J|Sx^^-Y^) and 7^^™'^(Sd|Sx^^yI.) 
satisfy the following property. 

Property 4: The sets 7^^"^(Ed|SxKyL) and 7^^°"''(I]d 
jSx^Y^) are not void if and only if Srf >~ E^ff |yt. 

Our result on inner and outer bounds of TLL{'Td\^x'^Y'-) 
is as follows. 

Theorem 3: For any '^x'<\y'-^ we have 

T^L^^H^dl^XKY^) ^ Ti''L^\^d\'^X'<Y^) 



KyL, 



Proof of this theorem is given in Section V. This result 
includes the result of Oohama 1201 as a special case by letting 
K ~ L and T = A = II. From this theorem we can 
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derive outer and inner bounds of TZl{T, D^\ Ej^/^yi,) and 
7?,L(r,Z?|I]j(-/<yi,) . To describe those bounds, set 



A 



conv< U 7e^"^(Ed|SxAyL, 
tr[rs<i'r]<_D 

7ef'(r,i?|i]xA-y.) 

convJ U 

I tr[rSdtr]<D 



A 



Set 



max I Ed 
rEd*rG5K(i3^) 
max I Ed I 

tr[rEd*r]<D 



A 



A 



Furthermore, set 

[r^>0: r(E-i + UE-i(^,)A)-"r £ 5a'(I?^)} , 

BL{r,D) 

[r^>Q: tr[r(E^i + 'AE^i^^,, A)-"r] < d] . 
It can easily be verified that 7^^°"'^(^, D^^\T,xkyl), 7^^"^( 

r, D^iEx^yi), 7^i™'^(^, die^^fO^ and ^(r, 

EjfKyi) satisfies the following property. 
Property 5: 

a) The sets 7^^'"'(^, D^lExKyi,) and 7^^°"*^(^, D^jEx^^ 
are not void if and only if > d^(rEx/f |yi,'r). 



YL) 



b) The sets 7e^"^(r,D|Exffyi) and 7e^°"*^(r,D|Exif y^) 



are not void if and only if I? > tr[rE_)(-A-|yt 'F]. 



c) 



ri'ei3i,(r,_D'f) 



conv < y 

nt'\r,D\j:x'<Y^) 



U 4™''(0(r,Ar^),r^|E^Ky.), 

r-f'ei3i,(r,Z5) 



7^f)(^,i?|E;,Kyz,) 

-conv I U 7^r'(r^)l. 

The following result is obtained as a simple corollary from 
Theorem [3] 

Corollary 1: For any F and any > d^(FExA |yi,'F), 
we have 

7^f '(F,Z?^|E;fKy.) C 7^f '(F,i?^|E;fKy.) 
C TZl (F, i?^ lExKy. ) C 7^i™*) (F, i?^ lE^Ky. ) ■ 

For any F and any D > tr[FExK|yt*F], we have 

7^f ' (F, D\^x'<Y- ) C Tlf (F, Z^lE^Ky. ) 

C 7^i(F,^?|ExKyO C 7ei™*HF,i?|E;fKyI,) . 



Those result includes the result of Oohama f 20| as a special 
case by letting K ~ L and V ~ A = I^. Next we compute 
6'(F, Z?,r^) to derive a more explicit expression of 7^^"'''(F 
, D\Yjxkyl )■ This expression will be quite useful for finding a 
sufficient condition for the outer bound 7^^"*^ (F , D\Y,x'<y^ ) 
to be tight. Let ai = ai{r^),i — 1,2,---,K be K eigen 
values of the matrix 

F ^ (e^k + MEjyi^^._f,jA^ *F ^ . 

Let ^ be a nonnegative number that satisfy 

K 



a. 



i=l 



Define 



u{T,D,r^)t\Trf[{[^^a-Y+a-'} 



i=l 



The function Ld{T, D,r^) has an expression of the so-called 
water filling solution to the following optimization problem: 



(4) 



K 

c.(F,D,r^) = |F|-2 max T]^- 

Then we have the following theorem. 

Theorem 4: For any F and any positive D, we have 

A more explicit expression of 7?.^°"*''(F, Z3|E YAy^-) using 
a;(F, Z), r^) is given by 

7^i°"*)(F,i5|E;,Ky.) 

^ U <°"')KF,Ar^),r^|Sx-yO- 
'■^eBi,(r,n) 

Proof of this theorem will be given in Section V. The above 
expression of the outer bound includes the result of Oohama 
1201 as a special case by letting K = L and T = A = 1^. 
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B. Matching Condition Analysis 

For L > 3, we present a sufficient condition for 7?.^"*^(r, 
^x^y^ ) ■ We consider the following 
condition on 9(r, D, r^). 

Condition: For any i G A, c^^^''d(r, D,r^) is a monotone 
decreasing function of > 0. 

We call this condition the MD condition. The following is 
a key lemma to derive the matching condition. This lemma is 
due to Oohama |fT9l. IITI. 

Lemma 1 (Oohama ^.^): If e{T,D,r^) satisfies the 
MD condition on Bl{ T,D), then 

Based on Lemma [T] we derive a sufficient condition for 
d(r, D,r^) to satisfy the MD condition. This sufficient con- 
dition is closely related to the distribution of eigen values of 



Define 



') , for i = 1,2, ■ ■ ■ , L. 



(5) 



From (|5]), we have 



2r,; = loe 



By the above transformation we regard 
and 9(r, D, r^) as functions of u^, that is, 

and 9{T,D,r^) = 9{T,D,u^). Let Uij be the entry of 

AT^^. Set di = [aiiai2 ■ ■ • ciix] ■ Let Q be a A' x A' unitary 
matrix. We consider the following matrix: 



L 



i=i 



For each i = 1, 2, • • ■ , i, choose the K x /v unitary matrix 
Q = Qi so that diQi = [||di||0- • -0] . For this choice of 
Q = Qj, set 

where w^j = ui - ■ ■ Ui-iUi+i ■ ■ - ul . Similar notations are 
used for other variables or random variables. Then we have 



If {i', i") ^ (1, 1), then the value of 



i=i 



does not depend on Ui. Note that the matrix 
has the same eigen values as those of 

We recall here that ai = aj{u'"),j = l,2,---,isr are K 
eigen values of the above two matrices. Let a,„in = ctminiu'") 
and Q!niax = «max(w^) be the minimum and maximum eigen 
values among aj,j = 1, 2, • • • , K. According to Oohama |fT9l . 
Il2n . we have the following lemma on those eigen values. 

Lemma 2 ( OohamanVi.HIl]!): For each i = 1,2,---,L, 
we have 



min(u ) < llOill^Ui + ?7i(M[j]) < amax('« ), 

K 



a 
du,. 



>0, forj = l,2,-.-,K, ^ 



dui 



The following is a key lemma to derive a sufficient condition 
for the MD condition to hold. 

Lemma 3: If Q;min(^i^) and Q!max('"^) satisfy 
1 1 1 



for i = 1, 2, • • • , i 



< 



12 1 



on i3i(r,L)), then 9{T,D,u^) satisfies the MD condition on 

BL{r,D). 

Proof of Lemma [3] will be stated in Section V. Let a*^g^^ be 
the maximum eigen value of 

From Lemmas [T][3] and an elementary computation we obtain 
the following. 

Theorem 5: If we have 



tr[rs 



tT]<D<^, 



then 



= nL{T,D\J:x'<Y^) = n^r'\^^D\ExKyL). 
In particular. 



mm 



■log 



Proof of Theorem |5] will be stated in Section V. From 
this theorem, we can see that if the value of D is very 
close to tr[rEx/f |yi.*r], 7^^"^(^, D\Exkyl) and 7^^°"'^(^, 
D\'E,xkyl) match. 
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Fig. 2. Distributed source coding system for L correlated Gaussian 
sources 



Let TZLi'^dl^Y'-) denote the set of all Scj-admissible rate 
vectors. 

We consider two types of distortion criterion. For each 
distortion criterion we define the determination problem of 
the rate distortion region. 

Problem 3. Vector Distortion Criterion: For given L x L 
invertible matrix F and > 0, the rate vector , R2 , 
■■■,Rl) is {T , D^)-admissible if there exists a sequence 
{((^i"\v'r\---,V:'i"\'/'("^)}5f=i such that 

limsupi?*"^ < R, , for i = l,2,---,i, 

n— >oo 

lim sup 



< A, for i = 1,2,---,L. 



Let 7^L(^, D^lEyi) denote the set of all (F, D^)-admissible 
rate vectors. The sum rate part of the rate distortion region is 
defined by 



IV. Application to the Multiterminal Rate 
Distortion Problem 

In this section we consider the multiterminal rate distortion 
problem for Gaussian information source specified with F^. 
We consider the case where K — L and A = 7^. In this case 
we have — + . The Gaussian random variables 
Yi,i = 1, 2, • ■ • , L are L-noisy components of random vector 
X^. The Gaussian random vector can be regarded as 
a "hidden" information source of F^. Note that [X^ ,Y'") 
satisfies Ys -> Ys^ for any 5 C A . 

A. Problem Formulation and Previous Results 

The distributed source coding system for L correlated 
Gaussian source treated here is shown in Fig. 2. Definitions 
of encoder functions (fi^i — 1,2, - ■ ■ ,L are the same as the 
previous definitions. The decoder function 0^"^ = 4'2^\ 
, 0^"'' ) is defined by 



a(«) 



:MiX---xML^X,i = l,2,---,K, 



where X is the real line in which estimations of Yi take values. 
_ ^ Y^^^ set 



1, Y2 


■) ' ' ' 






Y2 


A 


Yl. 





4 -E||y, 

7 A 



Y..\ 



^{Y,-Y,,Yj-Y,),l<i^j<L. 

Let Yiyl_y'^ ^ covariance matrix with dij in its (i, j) entry. 

For a given S^, the rate vector i?2, • • • , Rl) is S^- 
admissible if there exists a sequence ¥'2"'; ' ' ' > Vl""*' 

V'^"^)}^=i such that 

limsupi?^") < for i = 1,2,---,L, 
lim sup ii;yi_^L < . 



A 



mm 

e7?,t(r,_D^|Syi 



.1=1 



Problem 4. Sum Distortion Criterion: For given LxL invert- 
ible matrix F and D > Q, the rate vector {Ri, R2, ■ ■ ■ , Rl) is 

(F, D)-admissible if there exists a sequence {{ip^J^\ V2"^ ' 
("■ 



limsupiT!^") < R^, for i = 1,2, ■ • • ,L, 



lim sup tr [F(il]y^_^^)*r 



Let 7?.i(F, DjUyt ) denote the set of all admissible rate 
vectors. The sum rate part of the rate distortion region is 
defined by 



^^sum,L(r, D|EyL) = min 



Relations between 7^L(S<^|SYI,), 7eL(F, D^jSyi 
7?.i,(F, DjSyi,) are as follows. 



and 



7^L(F,7^^|I]y^) = U TeiCEdlSy^), (6) 

7lL(F,i?|Ey^) = U 7ei(I]d|I]yL). (7) 

tr[rEd'r]<D 



Furthermore, we have 

7eL(F,i^|SyL) = U 7ei(F,i^^|EyO. 



(8) 



We first present inner bounds of 7^.^(1]^ |Syi,), 7?.i,(F, Z?^ 
Eyi), and 7?.l(F, Z?|I]yi). Those inner bounds can be 
obtained by a standard technique of multiterminal source 
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coding. Define 



is a Gaussian 
random vector that satisfies 

for any C A and 

for some linear mapping 



and set 



A 



{ i?^ : There exists a random vector 
C/^ e ^(Sd) such that 

for any S* C A . } , 



7^f)(^,i?^|I]y.) 



A 

= conv 



7^f)(^,i?|Sy.) 



u 



rSd'Te5i,(D'^) 



A 

= conv 



u 



tr[rEd'rl<L> 



Then we have the following result. 

Theorem 6 (Berger and Tung /Ei/j-' For any positive 
definite E^, we have 

For any invertible F and any 13^ > 0, we have 

-^M^P^^Ljj.^^^ c 7^L(F,i:>^|Ei.L) . 

For any invertible F and any 13 > 0, we have 

'r!'^\t,d\y.y^) ^ 7^L(F,i:>|EyL). 

The inner bound 7^^"^(i:)^|EyL) for F = is well known 
as the inner bound of Berger ID and Tung ||5| . The above three 
inner bounds are variants of this inner bound. 

OptimaHty of 7t^'"'(L>2|I]y2) was first studied by Oohama 
(|9l- Without loss of generality we may assume that 



A 



where 

{(i?i,i?2) > ilog+ 

i?3-.>il0g[i] 

for some < s < 1 | . 



Since 7e*2(A|Ey2), i 
TZ2iD^\^Y^), we have 



1 , 2 serve as outer bounds of 



7^2(^'|Ey2) c 7^^ 2pi|Ey2) ^7^;,2(^2|Ey2). (9) 

Wagner et al. ifTOl derived the condition where the outer bound 
in the right hand side of (|9]l is tight. To describe their result 
set 

V = {{D^,D2):D^,D2>0, 

max{Di, 1)2} < min{l, p2 m\n{Di,D2} + 1 - p^}} ■ 

Wagner et al. ifTOl showed that if ^ V, we have 

7^2(i3' |Ey2) = 7^^ 2 |Ey2 ) H (£'2|Ey2 ). 

Next we consider the case of e T). In this case by an 
elementary computation we can show that 7^^'"'(D2|5^^^) 
the following form: 

7tr^(^^|Sy2) 



7^^2 |Ey2 ) n 7^;,2 (i?2|Ey2 ) n ni^ {d^\^y- ] 



where 



7^*,2(£'^|Sy2) 

= R2) ■ Ri + R2 

>ilog[(l-p2)^ 



D1D2 



A 



1 



(l-p2)2 



D1D2 



Ey2 = 



For i = 1, 2, set 



1 P 
P 1 

A 



The boundary of 7^2"^(^^l^y^) consists of one straight line 
segment defined by the boundary of TZ'^ 2{D'^\^y^) and two 
curved portions defined by the boundaries of TZl 2(^i|Ey2) 
and 7?,2 2(^2|Ey2). Accordingly, the inner bound estab- 
lished by Berger f?) and Tung (|5| partially coincides with 
7^2 {D^ I Ey2 ) at two curved portions of its boundary. Recently, 
Wagner et al. IfTOl have completed the proof of the opti- 
mality of 7^2™''(D^[Ey2) by determining the sum rate part 
^sum.2(£'^|Ey2). Their result is as follows. 

Theorem 8 (Wagner et al. fJUj): For any G T>, we have 



P e [0, 1) 



^sum,2(-D^|Ey2) 



mm 

(-Ri,-R2) 
6TC<'"'(_D^|Sy2 



7e,,2(A|Ey2) = IJ 7e2(i?'|Ey2). 

Oohama ||9l obtained the following result. 

Theorem 7 (Oohama For i = 1, 2, we have 

7e,,2(A|Ey2) =7e*2(A|Ey2), 



■log 



(1 



(-Ri 



R2 



D,D2 



According to Wagner et al. IfTOl . the results of Oohama IfTSlI 
and |fT6l play an essential role in deriving their result. Their 
method for the proof depends heavily on the specific property 
of L = 2. It is hard to generalize it to the case of L > 3. 
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B. New Partial Solutions 

In this subsection we state our results on the characteriza- 
tions of7^L(Sd|SyI.),7^L(^,i:'^|S]yI,), and TZl{T, D\Y.yl). 
Before describing those results we derive an important relation 
between remote source coding problem and multiterminal rate 
distortion problem. We first observe that by an elementary 
computation we have 

X^=AY^+N^, (10) 

where A ~ {^x^l +Sjyi)^^I]jyi and is a zero mean 
Gaussian random vector with covariance matrix Ejyt = (S^^-i 
+1]^^^)^^ . The random vector is independent of Y^. Set 



B 


A 






6^ 


A 


'([i?]ll,[S]22, 




B 


A 


rs'r, 






A 


*([B]ll,[S]22, 


■■,[B]ll). 



From ( [Tol l, we have the following relation between and 

L 



(11) 



where N is a sequence of n independent copies of N and 
is independent of Y^. Now, we fix {{ip']^\ (p'^\---, ip'"^\ 
arbitrary. For each n = 1,2,- ■■, the estimation 

X of X^ is given by 



x' = 



Using this estimation, we construct an estimation Y of Y 
by 



r =A-^X . 



which is equivalent to 



x' = Ay\ 



From (fTTl i and ( fT3T l, we have 



rL 



(12) 



(13) 



(14) 



Since is a function of Y , Y — Y is independent of 
N . Computing jj^x^-x^ based on (fT4l i. we obtain 



(15) 



From (fTSl) . we have 



-x^ I '^-^ 



B. 



(16) 



Conversely, we fix <y92"\ • • • : Vl'''' arbi- 



trary. For each n = 1, 2, • • •, using the estimation Y of Y 
given by 



y' = 



we construct an estimation X of X by (113b . Then using 
(fTTT i and ( fTST l, we obtain (fl4] i. Hence we have the relation ( fTSl ). 

The following proposition provides an important strong 
connection between remote source coding problem and mul- 
titerminal rate distortion problem. 

Proposition 1: For any positive definite E^, we have 

7^L(Sd|Sy•^) - 7^L(i(s]d + B)*i|Sxiy^) . 

For any invertible F and any > 0, we have 

nL{T,D^\Y.YL) =nL{TA-~\D^ +b^\i:x^Y^) ■ 
For any invertible F and any > 0, we have 

7^I,(^,i:»|I]yL) = 7^L(^i-^i:> + tr[i?]|I]x^y^')■ 

Proo/- Suppose that e 7^L(^(Sd + S)'i|I]xi-Yi )• 
Then there exists {((pj"-*, ip'^\ • • • , <p^"\ such that 

limsupi?'"' <R^, for i = 1,2,---,L, 

n— >CXD 

n— ^cxD 

Using X , we construct an estimation Y of Y hy Y = 
A^^X . Then from (fTSI l. we have 

limsupil]yi__^i, 

n— >-oo 

= limsupi^i - B 

which implies that i?^ e 7^L(A(S]rf + BfAlT^xLyL) . Thus 

7^L(Ed|SyL) D7^L(i(s<i + s)*i|S;fLy^) 



is proved. Next we prove the reverse inclusion. Suppose that 
G 7^L(Srf|EyI,). Then there exists • • • , <p^"', 

0^"^)}^=i such that 

limsupi?^"^ < Ri, for i = 1, 2, • • • , L , 

n—>-oo 

limsup ^^Y.Y^_^L ^ Erf . 

n— ^cxD 

- L - L r - L 

Usin^ Y , we construct an estimation X of X by X = 
AY . Then from ( fTSl ). we have 

lim sup iS^i .^t 

n— f CO 

= lim sup A 



t 2t 
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which implies that e 7^L(i(Sd + -B)*i iS^^iyi) . Thus, define 



is proved. Next we prove the second equality. We have the 
following chain of equalities: 

rSdtrG5i,(£)^) 

y 7ei(i(S]<i + B)*i|Ex-y.) 

-rB'rG5i,(D^) 
ri-^A(Ed+B)*l*(rl-i) 

y 7^L(Ed|E;f.y.) 

= 7^L(^i-^ + b^is^Lyi,) . 

Thus the second equality is proved. Finally we prove the third 
equality. We have the following chain of equalities: 

7^L(^,^?|EyL) 

y 7^L(Srf|I]i..) 

tr[rSd'r]<D 

y 7^L(^i(Ed + B)'i|S];f^yz,) 

tr[rs<j«r]<n 

y 7^i(i(Srf + S)'i|S];,LyO 

tr[ri-^A(Ed + B)'i*A""r] 

-tr[rB'r]<z> 

y 7^L(i(Sd + B)*i|E;f^yL) 

<L>+tr[B] 

Ed=A(Sd+B)'A^E^.t|^i,, 
tr[rA"iErf'(rA"i)]<Z5+tr[S] 

= 7^L(^i-^i:> + tr[S]|Ex^y^)• 

Thus the third equality is proved. ■ 
Proposition[T]implies that all results on the rate distortion re- 
gions for the remote source coding problems can be converted 
into those on the multiterminal source coding problems. In the 
following we derive inner and outer bounds of 7^L(Sd|SyL), 
nL{T,D^\T.YL), and nL{T,D\T,YL) using Proposition [U 

We first derive inner and outer bounds of TZL{T,d\T,YL). For 
ri > 0, i Q A, let Vi{ri), i G A he L independent Gaussian 
random variables with mean and variance (t^./(c^''' — 1). 
Let I]yt(^t-) be a covariance matrix for the random vector 
V^{r^). Fix nonnegative vector r^. For 9 > and for S* C A, 



A 



rs=0 



y-l I y-1 



Set 



Syi +Sv'ic(rsc) 



iL(I]d)"|r^>0 
Define four regions by 

<°"*)(0,r^|Ey.) = {R^ : ^ i?, > J5 (0, rslrsO 



A 



for any 5 C A . } , 

<°"*)(S,|Ey.)- y <°"')(|S, + i?|,r^|Sy.), 

r^e^t(Ed) 

for any 5 C A . } , 

7efHSrf|Sy.) = conv| y 7^f)(r^|Sy.) 

The functions and sets defined above have properties shown 
in the following. 
Property 6: 

a) For any positive definite S<j, Gi^d) = GiA{'^d + Bf A). 

b) For any positive definite T^d, we have 

7^f' (s.lSy.) = n^t\M^d + BfAi^xr-Y^) ■ 

c) For any positive definite T,d and any C A, we have 

Js(|Sd + B\,rs\rs^) = IsilM^d + rs|rsc), 
Jsirs\rs<=) = Jsirs\rs-)- 

d) For any positive definite E^, AL{^d) = AL{A{Y,d + 
BY A). 

e) For any positive definite Ed, we have 

7e^"'(Ed|Sy.) = 7e^"'(i(Ed + bya\j:x-y-) ■ 

From Theorem |3] Proposition [T] and Property |6] we have 
the following. 

Theorem 9: For any positive definite E^, we have 

7?-i"^(Srf|Eyt) C 7^^"^(Erf|Eyi) 

c nu^dl^Y^) c <°"*^(Ed|Eyi) . 
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Next, we derive inner and outer bounds of TZl (F,!)^ |EyL ) 
and nL{T,D\Y.YL). Set 



9{T,D^,r^) = max \T.d + B\ , 
rEi'rG5i(D^) 
6'(r,i:>,r^) = max + B\ . 

tr[rSd'T]<D 

Furthermore, set 

{r^ > : F(E^i + E^i^^.^-^F e 3^0^^)} 
Bl{T,D) 



A 



= > : tr 



-iti 



Define four regions by 

7e^°"*)(F,i?^|EyL) 



A 



ri-eei,(r,Di') 
7^f^(F,i?^|E^.) 



A 



A 



A 



conv<^ y Tef^r^lEy.j 

y <°"'^(0(F,Ar^),r^|Sy-) 

r^eei,(r,D) 

7^f)(F,i?|E^.) 

convJ y 7^f)(r^|Ey.) 
[ri-eBLir.D) 

It can easily be verified that the functions and sets defined 
above have the properties shown in the following. 
Property 7: 

a) For any invertible F and any Z)^ > 0, we have 

7^f)(F,i?^|Ey.) 

For any invertible F and any Z? > 0, we have 

= n^i''\TA-\,D + tr[B]\T,xLYi-). 

b) For any > 0, we have 

Ed e A{r^) <^ i(Ed + BY A e A{r^) , 

-2 



0(F,i?^,r^) 



A 



6i(F^-\i:)^,r-^), 



e{T,D,r^)^ A 0{rA-\D,r^). 



c) For any invertible F and any D > 0, we have 



7e^°"'^(F,Z?^|EyO 



7er(F,7^^|Ey.) 
= nP{TA-\D^ + b^l^xLYL) . 
For any invertible F and any Z) > 0, we have 

= <°"*)(Fi-i,Z? + tr[S]|E;,.y.), 

7^f>(F,i^|Ey.) 

= 7^^"^(Fi-^i:> + tr[i3]|E^Y^-F^) ■ 

From Corollary [T] Proposition [T] and Property |2l we have 
the following theorem. 

Theorem 10: For any invertible F and any I? > 0, we have 



C 7^L(F,I?^|EyO C 7ei°"*^(F,Z?^|EyL) . 

For any invertible F and any 13 > 0, we have 

7ef^(F,D|EyL) C7^f)(F,Z3|EyL) 
C 7^L(F,D|Ey^,) C 7e^°"*^(F,D|EyL) . 

Next, we derive a matching condition for T?,^""'-' (F, DjEyL ) 
to coincide with 7^^'"^ (F, D|Eyi. ). By Theorems |5] and [TOl 
Proposition [U and Property |7l we establish the following. 

Theorem 11: Let yU*^;^ be the minimum eigen value of 



(in)/ 



B==F(E 



E^L E^"^ EjY-c- ) 'F . 



If we have 

0<D<{L + - tr [F(E^.. + E^..E^iEjv.)'F] , 

then 

7^f)(F,i5|Ey.)=7^f'(F,Z3|Ey.) 
= 7^L(F,D|Ey^.) = 7^^°"*^(F,i:)|EyI.) . 

We are particularly interested in the case where F is the 
following diagonal matrix: 

/ 



71 



72 







0\ 



7L / 



i=l 



1, 



(17) 



Let 5 > be an arbitrary positive constant specified later. We 



choose E^L so that E^i 



A 



we have B 



5^T^. Set T^xL = FE^lF. Then, 



51 L + XL ■ ^^^^^ 

we have 



>5. 



(18) 



Let Aniin be the minimum eigen value of Eyt- Since E^t >- 
have t-^\ -< A,;J„F-2. Hence we have 



tr 



(19) 
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where the last equahty follows from the choice of F specified 
with iTT) . From ( fTSl l and (fT9] l. we have 



{L + - tr[B] > (L + 1)5 - tr 



(5/r + S^E-] 



XL 



>{L + 1)6 -L6- S^X-i^ 



S - S^X 



2\-l 



Hence if 



< D < 6 - S^X 



2 \-i 



(20) 



(21) 



then the matching condition holds. The right member of (ISTT i 
takes the maximum value |Ai„in for 5 = ^X^in- Summarizing 
the above argument, we establish the following corollary from 
Theorem [TT] 

Corollary 2: If the minimum eigen value Amin of S 
satisfies 

0<D< ^Aniin, 

then for any diagonal matrix F specified with (fTTI i we have 

7^i'"^(F,i?|s]yL) = 7e^"'(F,i?|i]yL) 



C. Sum Rate Characterization for the Cyclic Shift Invariant 
Source 

In this subsection we further examine an explicit charac- 
terization of i?sum,L( D\Yjyi) when the source has a certain 
symmetrical property. Let 

1 2 ■■■ i ■■■ L 
r(l) t{2) ■ ■ ■ t{{) ■ ■ ■ t{L) 

be a cyclic shift on A, that is, 

t(1) = 2, r(2) = 3, • • • , r(L - 1) = L, t{L) = 1 . 

Let px^xa) = PXiX2---Xi^{xi,X2, • ■ • ,xl) be a probability 
density function of X^. The source is said to be cyclic 
shift invariant if we have 

PxA^t(,A)) = PXiX2---Xl{x2,X3, ■ ■ ■ ,XL,Xi) 

= PXiX2---Xi,(a;i,X2, ■ • ■ ,xl~i,xl) 

for any {xi,X2, ■ ■ ■ ,xl) G A"^. In the following argument we 
assume that satisfies the cyclic shift invariant property. 
We further assume that Ni,i G A are i.i.d. Gaussian random 
variables with mean and variance e. Then, the observation 
= X'" + N'" also satisfies the cyclic shift invariant 
property. We assume that the covariance matrix S^t of 
is given by e/^. Then A and B are given by 

A = (eS^l +lLy \ B^e {II + eS^i) • 

Fix r > 0, let iVi(r), i £ A be i i.i.d. Gaussian random vari- 
ables with mean and variance e/ (1 — e^^''). The covariance 
matrix I]jv''(r) for the random vector A^^(r) is given by 



1 - c 



-2r 



Let Ai,i G A be L eigen values of the matrix E^^i, and let 
Pi — Pi{r),i G A be L eigen values of the matrix 



*i ( E^i + ^-^—Ij. 1 A. 



Using the eigen values of Ej^t, /3i{r),i G A can be written 
as 



A, 



A; + e V A, + e 



A, 



-2r 



Let ^ be a nonnegative number that satisfies 

L 

J2m~Py]++l3r'} = D + tr[B]. 



i=l 



Define 



i=l 

The function uj{D, r) has an expression of the so-called water 
filling solution to the following optimization problem; 

L 



Co{D,r) 



Set 



Jp,r) = i log 



max 

Ci0i>i,ieA, 



e^-^n^y^ +B\ 



(22) 



i=l 



C(r) ^ tr 



A-^ { E^i + ^^/z 



'A- 



By definition we have 



(23) 



Since is a monotone decreasing function of r, there exists 
a unique r such that ^(t') = £'+tr[i?], we denote it by r*{D+ 
tv[B]). Note that 

{r,r,---,r)^BL{A-\D + iY[B]) 

L 

^ Q{r) <D + tr[B] ^r>r*{D + tr[B]) , 



w(i:»,r*) = \A\- 



1 - e 



Set 



RZ-,LimYL)= min J(D,r) 

r>r* (_DH-tr[BJ) 



Then, we have the following. 

Theorem 12: Assume that the source X^ and its noisy 
version = + are cyclic shift invariant. Then, we 
have 

Proof of this theorem will be stated in Section V. 
Next, we examine a sufficient condition for i?'^''' , (D 
Eyi) to coincide with Rsum.Li -D|Eyi,). It is obvious 
from the definition of J_{D,r) that when e~'^'"^'uj{D,r) is a 
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monotone decreasing function of r E [r*{ D + tr[B]), +co), 
we have ) = i?sum.L(£'|Syi, ). Let A be the 

maximum eigen value of • Set 

Ao ('') = ijiin A(r) , Ai (r) = max /3j(r) . 

l<t<L l<i<L 

Then we have the following two lemmas. 
Lemma 4: If 



or equivalent to 



2r 



L-l 



A„ 



Aj, + e Ai, 



. 2r 



Xi„ + e A, 



< 



L 



L- 1 



An 



A„ 



,2r 



K + e 



(24) 



holds for r > r* {D+tr[B]), then c~^^''(D(_D, r) is a monotone 
decreasing function of r e [r*{D + tr[i?]), oo). 
Lemma 5: If we have 



A,; 



A,:, 



< 



Ail + £ K 
4L 



i - 1 



An 



K, + e 



Ai, + e 



(25) 



then the sufficient condition (|24] | in Lemma |4] holds for any 
nonnegative r. 

Proofs of Lemmas |4] and |5] will be given in Section V. If 
we take e sufficiently small in dZSl ) in Lemma |5] then the left 
hand side of this inequality becomes close to zero. On the 
other hand, the right hand side of dZSl l becomes close to 77^. 
Hence if we choose e sufficiently small, then the inequality 
(|24] | in Lemma |4] always holds. Next we suppose that the 
Gaussian source satisfies the cyclic shift invariant property. 
It is obvious that for arbitrarily prescribed small positive e, 
we can always choose a Gaussian random vector so that 
^ elL and = X^+N^. For the above choice of N^, 
the Gaussian remote source also satisfies the cyclic shift 
invariant property. Summarizing those arguments we obtain 
the following theorem. 

Theorem 13: If is cyclic shift invariant, then r2},„ r (D\ 

S U 111 . ^ I 

Syt) = i?sum,L(^|Syi,). Furthermore, the curve R ^ 
Rsnn\,L{D\YiYL) has the following parametric form: 

L 



R 



D = 



1 



■log 



i=l 



L 

E 



- iv[B] . 



V. Proofs of the Results 

A. Derivation of the Outer Bounds 

In this subsection we prove the results on outer bounds of 
the rate distortion region. We first state two important lemmas 
which are mathematical cores of the converse coding theorem. 
For i = 1, 2, • ■ • , L, set 

1 



(26) 



For S* C A, let Q5 be a unitary matrix which transforms 
into = QX^. For 



X' 



(X^(1),X^^(2),...,X^H) 



we set 

Z'' = QX" = (QX^(l), QX^(2), • • • , QX^(n)) . 

Furthermore, for X^ = (^^'(1), ^^(2), ■■■,X^{n)), we 
set 



Z =QX = (QX^(l),QX^(2),---,QX^(n)) 

We have the following two lemmas. 

Lemma 6: For any i ^ 1,2, ■ ■ ■ , K, we have 



hiZ.,\Z^^W^)<hiZ.,-Z,\Z^^-Z^^) 
4log{(2.c)[Q(iE-_,.,)*Q]-], 

where h{-) stands for the differential entropy. 
Lemma 7: For any i = 1,2, - ■ ■ ,K, we have 



hiZ^lZ^^W"^) 



> - log <^ (2^c) 



Proofs of Lemmas |6] and [7] will be stated in Appendixes 
A and B, respectively. The following corollary immediately 
follows from Lemmas |6] and [T] 

Corollary 3: For any 'Sx'^y'- ^nd for any </'2"^ ' ' ' > 

yj^"'', V'*'"')^ we have 



ME 



A'A(ri"') 



A. 



From Lemma |7j we obtain the following corollary. 

Corollary 4: For any S* C A, we have 



I{X'';Ws) < - log 



A 



(27) 



Proof: For each i £ A — S", we choose W^i so that it takes a 
constant value. In this case we have rj"'' = for i € A — S. 
Then by Lemma |7] for any i — 1,2, ■ ■ ■ , K, we have 

h{Z,\Z^^Ws) 



> -loga2^c) 



Ns{r 



■(28) 



We choose a unitary matrix Q so that 
becomes the following diagonal matrix: 



Q(^-^i^+^A^2ir^^,A]^Q 



Ai 



A, 







0\ 



Xk J 



(29) 
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Then we have the following chain of inequalities: 



IiX'';Ws) = HX") - hiX'^lWs) 



-K\ 



(a) 



h{X'')~h{Z''\Ws) 
K 

< h(x^)-Y,h(z,\Z^^Ws 



(b) n 

< -log[(2^e)^|S;,.|] 



1=1 

\K I 



(c) n 



K 



-log|Sx/.| + }_^-logA, 



n 



Ns{r' 



A 



■log 



Step (a) follows from the rotation invariant property of the 
(conditional) differential entropy. Step (b) follows from ( l28T l. 
Step (c) follows from dig. ■ 

We first prove the inclusion TZL{J^d\ ^x^y^ 
SxA'yi,) stated in Theorem[3] Using Lemmas |6]|7] Corollary 
|4] and a standard argument on the proof of converse coding 
theorems, we can prove the above inclusion. 

Proof of TZLi^dl^x'^Y^ C 7e^°"*)(I]rf|Ex«'Y^)- We first 
observe that 

Ws ^Ys ^ X^ ^ l^sc ^ Ws<= (30) 
hold for any subset S of A. Assume (i?i,i?2, ■■■,Rl) G 
_^(»)^^(n)|oo^^ such that 

limsupi?^^"' < e A 
limsup -S^^,_^A' ^ 

n— >-oo 

We set 



TZlC^^ciI'^x'^y^)- Then, there exists a sequence {((^^"^(^2"^ 



(31) 



ri = limsupr^-"^ ~ limsup —I{Yi;Ws\X^) . 



For any subset S" C A, we have the following chain of 
inequalities: 



> J2h{W,)>H{Ws\Ws^ 



les 



= liX^'^WslWs^ 
liX^'^WslWs^ 



H{Ws\Ws^X'') 
Y,H{W,\X^) 



ies 



(b) 



(c) 



/(X^^ ; Ws I Ws^)+Y, HiW^lX"") 



I{X^^;Ws\Ws^)+n^r. 



(") 



(33) 



where steps (a),(b) and (c) follow from ( l30b . We estimate a 
lower bound of I{X^; Ws\Ws'^)- Observe that 



; Ws\Ws^) = ; W^^) - ; Ws^) (34) 

Since an upper bound of /(X5C ; Ws" ) is derived by Corollary 
m it suffices to estimate a lower bound of I{X^ ; W^). We 
have the following chain of inequalities: 

I{X'^- W^) = h[X^) - h{X^\W^) 
> hiX'^)-h{X'^\X^) 



> h{X'^)-h{X'^ -X") 
>^log[(2^e)^|S;fK|]-^log [(27re) 



K 



= flog 



^XK\ 



n^X'<-X" 



(35) 



Combining ( l34l i. ( |35] |. and Corollary 21 we have 



IiX'^;Ws\Ws^)+nJ2 



„(") 



n 

> — log 
- 2 8 



— Ids 

2 ^ 



n 



I + Y^xK^AY.-^ 



n 



Q2r 



X^ AT„^fJ">\ 



n^X^~x' 



Note here that I{X^; Ws\Ws<:)+n J^ies ^j'"^ nonnegative. 
Hence, we have 



IiX'';Ws\Ws^)+nY,r'i"^ 



> nj_g 



r, -^y/f -e-A I ,rc. I r 



Combining ( |33] | and ( |36] |. we obtain 



(36) 



(32) 



In) 



(37) 



for 5 C A. On the other hand, by Corollary [5] we have 



y-l 



XK 



JVa(»- 



1 



(38) 



By letting n — !> 00 in (|37] | and (|38] | and taking dJTT i into 
account, we have for any 5 C A 



and 



(39) 



(40) 



From ^ and (gOll, 7^L(Sd|SxA■yI.) C 7^i°^(I]dlSxAF^.) 
is concluded. ■ 
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Proof of Theorem^ We choose a unitary matrix Q so that and for ^ = 0, 1, • • • , i — 1, set 



ai 



OL2 











aK 



Then we have 



In particular, 



r^"? = = ^I{Y,;W,\X,), for i e A. 











For e A{r^), set 

Erf ^ Qrs/r^Q , 

Since 

(|4T]) , and tr[rS]d'r] < D, we have 

6 > a^^ for I 1,2,---,K, 

^e:=tr[sJ =tr[rE/r] <D. 



Furthermore, set 



(41) 



(") ~ iJ"^ ... J^'h for/-0 1 ■•■ L-1 

t'{A) ~ y' 1,1 ' ' 1,2 ' I '/,L j 7 I — u, i, , i 

(«) A _l_ („) 

1=1 



By the cyclic shift invariant property of X\ and Y/^, we have 
for / = 0,1, 1, 



1 Vr(") _ i Vr^'^ 



(") _ M 



For Erf = [djj], set 



(42) 



L-l 



(45) 



(=0 



Furthermore, by Hadamard's inequality we have 

K K 

lErfi = irr^is^i < |r|-2[][Erf],, = irr^ (43) 

i=l i=l 

Combining ( |42] | and ( |43] |. we obtain 

= max I Erf 
tr[rSd'r]<D 



Then, we have 



1 

limsup-^iE^^_y 
1 

lim sup — i E 



(a) 



A' 



(=0 



< 



\-^if . ,„ i=l 



(46) 



The equality holds when Erf is a diagonal matrix. ■ 
Proof of Theorem 172} Assume that (i?i,i?2, • ■ • tRl) G 
TZLiD^^Y^)- Then, there exists a sequence {{ip^^\ip2^\ 
• ■ • , <<5i"\ 0^"H^=i such that 

limsupi?f'^ < e A 

n— >cjo 

limsup -Ey^ y^ ^ Erf, tr[Erf] < D 

n— >oo ^ 

for some Erf. 



Step (a) follows from the cyclic shift invariant property of 
Y !y. Step (b) follows from ( l44l i. Step (c) follows from the 
definition of Erf. From Y a, we construct an estimation Xa 
of Xa by Xtv = AYk . Then for Z = 0, 1, • • • , L - 1, we 
have the following. 



(") ^ — 



(A) 



(A, 



(44) 



(b) ^ ^ 

— ri X . X 1 



W I5.-I 



'i + E 



'(A) 



Xa|Ya 



(47) 



For each Z = 0, 1, • • • , L - 1, we use y^^^^'j)' ' ' ' ' 

^^t\l)^ for the encoding of (l^i , 1^2, ■ • • , ^l)- For i e A 



Steps (a) and (c) follow from the cyclic shift invariant property 
of ATa and Xa, respectively. Step (b) follows from Corollary 
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[3] From ( |47] i. we have 

L-l 



(a) 



^a|1a 
XaI>a 

H -1 



B M 



(=0 



(48) 



Step (a) follows form that (AS^A+SxaIVa) ^ convex with 
respect to S. On the other hand, we have 

1 



1=0 L 



(A)' 



1 - 



(49) 



Step (a) follows from that 1 — c is a concave function of 
a. Combining (l48T l and ( |49] l, we obtain 



1 - c- 



"Xa 



from which we obtain 

L-l 



1 -c 



(50) 



Next we derive a lower bound of the sum rate part. For each 
Z = 0, 1, • • ■ , L~l, we have the following chain of inequalities. 

ieA isA isA 

= /(XA;iy.<(A)) + ^(^rHA)I^A) 

HXA^Wr^^^)) + Y,H{Wu\Xa) 
= /(Xa;M^.<(a)) + E^(^a;T^mI^a) 

(b) 



(c) „ 
> 2 log 



n Xa — X I 



loa 



2 ^ 







S^aI^^a 




A 




'i + SxAlYA 



(51) 



Step (a) follows from ( [30l l. Step (b) follows from ( |46] l. Step 
(c) follows from ( [35] ). From ( ISTT i. we have 

IS^A+^I 



iGA 



- L ^2 

i=i 



(a) 1 , 
> - loa 
- 2 ^ 



^Ya 



B\ 



1 

;=i 



Lr^"K (52) 



Step (a) follows from that — log |S] + i3| is convex with respect 
to S. Letting n oo in (ISOl l and ( l52t and taking ( |46] | into 
account, we have 



ieA 



Lr. 



1 -c- 



Il>A 



trpd + B] = tr[E<j] + tr[B] < £» + tr[B] . 
Now we choose a unitary matrix Q so that 



(53) 

,(54) 
(55) 



/?2 











/3i 



Set 



A 



A 



From ( |54] i and ( |55] ) we have 

L 

^ ^ tr [Srf + b] = tr [Ed + S] < B + tr[B] 

i=l 

From ( [56] l, we have 

L L 

i=i ''^^'^ ' i=i 

<^r > r*(i:' + tr[B]) . 



(56) 



(57) 



Furthermore, by Hadamard's inequality we have 

L L 

iSd + B\^ \td + B\< l[[td + B]u = n (5^) 
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Combining ( |56] l and ( |58] ), we obtain 



|Sd + S|< max ffe^ =<:^(7^,r). (59) 

Hence, from (|53] l, (|57] i. and ( |59] l we have 



E_Ri > min - los 
r>r'(n+tr[S]) 2 

i—1 



= min J_{D,r) = Rsn-m,L{D\^Y'-) ^ 
completing the proof. ■ 

B. Derivation of the Inner Bounds 

In this subsection we prove 7i}-™\Y,d \Tixkyl) C Tl]^(T,d 
|Exifyi) stated in Theorem |3] 
Proof of 7e^"^(Sd|SxA'Yi.) C TZL{^d\^x^Y^)-- Since 



TZ^lf'i Y,d\£'x"^Y'-) ^ T^L{'^d\^x"^Y^) is proved by Theorem 
[U it suffices to show 7e^"^(Ed|I]xicyi,) C tI' 

to prove 7^^"^(I]d|ExA-yt) C 7^L(Sd|Ex^^y^.)■ We assume and set 
that S 72.^™^ (EdlSxA'yi,). Then, there exists nonnegative 
vector such that 



and 



^ i?, > ii'lrslrsc) for any 5 C A . 



(60) 



Let Vi,i G A be L independent zero mean Gaussian random 
variables with variance ay. . Define Gaussian random variables 
Ni + Vi. By definition it is obvious 



Ui, i e A by f/i = Xi 
that 

for any 5 C A . 



(61) 



For given > 0, i G A, choose cry. so that cry. = ct|^. / (e^''' — 
1) when ri > 0. When r.i = we choose Ui so that Ui takes 
constant value zero. In the above choice, the covariance matrix 
of A^^ + becomes S^tj^t). Define the linear function ip 
of by 



Set X^ = i' (U^) and 



-It 4y-l 



d,,: - E 



d,, = E 



ATi — X., 



,l<i^ j <K. 



Let E 



be a covariance matrix with dij in its 



entry. By simple computations we can show that 

,A)-^ ^ Ed 



'AT, 



(62) 



and that for any 5 C A, 

JsirsM=I{Ys;Us\Us^). (63) 

From (|6B and (|62]i, we have G ^(E^). Thus, from (|63] l 
72^'"^(Ed|ExAyi^) C 7^^"^(Erf|ExicyI,) is concluded. ■ 



C. Proofs of the Results on Matching Conditions 
We first observe that the condition 

-1 



tr 



< D 



is equivalent to 



(64) 



Proof of Lemma |5} Let A = {1, 2, • ■ • , and let S" C A 
be a set of integers that satisfies > C in the definition of 
e{r,D,u^). Then, d{r,D,u^) is computed as 

X-[S| 



1 



Vies 



Fix i G A arbitrary. For simplicity of notation we set 

,L 



* = log - log 0(r, 7^, 7.^) , 



Computing the partial derivative of ^' by u;, we obtain 



dui 



i6S 



J_ _ K-\S\ 1 



4-||di||2 + - (lla^lpuj + 



daj_ 



/ J a- 



a,; 



From Lemma |2] and ( |65] |. we obtain 



(65) 



dui ~ \ dui 



1_ K - \S\ 1 



kes 



To examine signs of contents of the above summation we set 



"'^-E--^Hx.-«..) 
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If |5| = /v, $j > 0,j' e A is obvious. We hereafter assume 
15*1 < K — 1. Computing (f>j, we obtain 



> X 

(a) 
> 



J6S 






- \s\ 










jes ■> J 




K - \S\ 


".J 




/\ 


-\s\ , 



Proof of Lemma ^ We first derive expression of ui{D,r) 
using Pi ~ fii{r), i G A. Let S* be a set of integers that 
satisfies /?~^ > C in the definition of Cj{D,r). Then Q{D,r) 
is computed as 



cI;(D,r) 



(L-\S\)^ 



L-\S\ 



Fix i E A arbitrary and set 



{Xi Q^min) 



A 



X^ J2 

if-IS"! K-\S\ , 

> Xi ■ — : • (Xi - amin) 



"ii = Lr - \oguj{D,r) . 
Computing the derivative of 5* by r, we obtain 



X^iK^\S\) 



1 1 1 



(66) 



Step (a) follows from the inequality (|64] |. From ( |66] |, we can 
see that if 

1 



1 1 . 

< — for z e A, 



Q!min(''-^) amax(f-^) X' 

then, $j > for j £ S* . 

Proof of Theorem |5} By ( |64| |. we have 



dr 

Zees 



fees 



e V Afc + e 



J_ L-\S\ 1 

fees 

1 ^-15"! 1 



fees 



L 

W\ 



1 /v - 1 

< D 



amm(?' ) amax(r-^) 
1 



To examine signs of contents of the above summation we set 

2 ( 



K 



A e-^'-|'?l / A 



Hence, if 



or equivalent to 



amax(?-^) amax(r-^) 



D ^< ' 



eL V Afc + e 



"max (7-^) Xt 



D-—] amax(r^) < K 

Xt 



fees 



(67) 



If \S\ = i, $fc > 0, fc G A is obvious. We hereafter assume 
|5| < L — 1. Computing we obtain 



holds for G ^^(r, Z?) and i G A, the condition on an 
and ttmax in Lemma |3] holds. By Lemma |2] we have 



$fe 



c-^1^| f A 



amax(r^) < «,;ax G Bl{T,D). 

It can be seen from ( |67] | and ( |68] | that 
1 



"max < for i G A . 



(68) 



(69) 



eL V ^fe + <^ 
6-2^51 / Afe 



/3fe i^-E 



fees ' 



eL \ Afe + e 



is a sufficient condition for ( |67] | to hold. By Lemma |2] we 
have 

|2^ , „j„.L ^ ^ „, /„,L^ 



eL V ^fe + <^ 
e-^'-\S\ [ Xk 



L-\S\ 
/3fe 

^/^Jf E 



v,feeA-s 



/3fe 



Xi = ||ai|l -2-+'7i("H)< lim amax(u ) 



> 



eL VAfe+e 



L-\S\ 



< a*^^^ for i G A, 



eL VAfe + e 
c-^'-\S\ f Xu 



L-\S\ 



from which we have 



D 



Xt 



max — max 



Thus, if we have Da*^-^^^^ — 1 < JiT or equivalent to L* < 

{K + l)/a*^^^, we have 



eL V'^fe+^y /3»o 
Step (a) follows from 

^-E^^o-^-E^^ E 



(70) 



fe=i 



kes feeA-s 
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From ( TTOI ). we can see that if 



eL \Xk + e 
^ns\ ( A, 



An 



A, 



eL \Xk + £ J Ao 

2 



Appendix 

A. Proof of Lemma |6] 

In this appendix we prove Lemma |6l To prove this lemma 
we need some preparations. For i e A, set 

A 



A, 



1 1 



> 0, (71) 



F,mQ) 



sup 



Ai eL \Xk + eJ \f3i„ 
then $fe > for /c G A. The inequality ( ItTI i is equivalent to 
1 1 



Ao Ai 



< 



Afe J \S\ A, 



Hence 



1 1 ^ pmax + ^ y eC^'-^Ao ^72) 



Ao Ai V 



i - 1 Ai 



is a sufficient condition for $fc > 0, fc G A. The condition 
is equivalent to 



AiM-AoW<ec^"- 

completing the proof. 
Proof of Lemma \5}i Set 

F{r) 



L fx 



L-1 \ X 



(AoM)' 



A 



,2r 



Aio + e Aij + e 



A,o + e 



Then, the sufficient condition stated in Lemma |4] is equivalent 
to 



A., 



A„ 



< 



A,i + e Aj„ + e 

L ( Ajiiax + ( 



L- 1 



Ain + e 



F{t) . (73) 



To derive an explicit sufficient condition for ( |73] l to hold, we 
estimate a lower bound of i^(r). Set 



T(r) = e^"- - ^ 



Ail 



Aio + e Ail + e 



An + e 



Then 



pi 



F{r) = [T{r)]-^[T{r) + PY = Tir) + —— + 2P 



T{r) 



> 4F = 



4A,; 



Hence, 



< 



All + e 



Aji + e Ai(, + e 
4L /A 



A„ 



A,,; 



L ^ 1 \ Aniax / \Aio + £/ \Aii + e 

is a sufficient condition for ( |73] ) to hold. 



To compute i^i(S|(3), define two random variables by 

Note that by definition we have Z^ = QX^ . Let Pxkxk 
{x^,x^) be a density function of (X^,A"^). Let Q^k^k 
{z^ , z^) be a density function of {Z^ , Z^) induced by the 
unitary matrix Q, that is, 

Iz'"^ (^^: -2^) — P'^QZ'^'^QZ"^ CQz^ , *Qz^) . 

Expression of Fi{J^\Q) using the above density functions is 
the following. 

= sup h{Z,\Zl^,) 



PxK^x 



sup 

°xK^xK ■ 
S ^ r.' S 



The following two properties on Fi{Y,\Q) are useful for the 
proof of Lemma |6] 

Lemma 8: Fi{Yi\Q) is concave with respect to E. 

Lemma 9: 

i^,(I]|Q) = ilog{(2^c) [QE'Q]:'} . 

We first prove Lemma |6] using those two lemmas and next 
prove Lemmas [8] and |9] 

Proof of Lemma^ We have the following chain of inequal- 
ities: 

h{Z,\Zf^W'') 
< hiZ, - z, I - Z^) 

<j2Hz.{t)-z,it)\z^^it)-z^0) 



t=l 

(a) " / 

t=l 



(b) 

< n 



Q 



\ t=i 



(t)-i:^(t) 



Q 



Q 



(c) 71 
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Step (a) follows from the definition of Fi{T,\Q). Step (b) 
follows from Lemma [8] Step (c) follows from Lemma |9] ■ 
Proof of Lemma |S} For given covariance matrices S'*'' 
and T,^^\ let p^^L,^^ and p^^j^.^u: be conditional densi- 

ties achieving Fi{Y,^^^\Q) and ^^(E^^^IQ), respectively. For 
< a < 1, define a conditional density parameterized with a 
by 



(a) 



n ^ (0) , (1) 



Let P^'-^K-^K be a density function of (X^ ,X^) defined by 
ip^^K^xJ^' P'x'<)- ^ covariance matrix computed 



(q) 

from the density p^^- Since 



PXK (1 - 0!)PxK + O^PxK 



„(0) 



.(1) 



we have 



(0) 



(74) 



Let q^^K^K be a density function of {Z'^ , ) induced by the 
unitary matrix Q, that is. 

By definition it is obvious that 

(a) /I \ (0) I (1) 



Then we have 



(l-a)F,(E(o)|Q)+a^^,(I](i)|Q) 



(a) 

< 



„(") 



(b) 



Q 



Step (a) follows from log sum inequality. Step (b) follows 
from the definition of Fi{J^\Q) and (|74li. ■ 

Proof of Lemma ^ Let 



and let 



- (G). 



be a conditional density function induced by '?^^(')- We first 



observe that 



) (G) . , ^ dz^ ^ • (75) 

From ( ITSI i, we have the following chain of inequalities: 

= - / g^,,(z^')logg^^|^^^(z,|z[^)dz^ 
g^.-(z^)loggg)^^^(z,|z[^)dz^ 



< 



q^.-(z^^)loggfJ(z^)dz^ 



9^..(z^^)logg^-^(zf])dz 



(a) 



4^j(z^^)loggfj(z^^)dz^ 



2 if < 



4"j(z^)logq^^(z^)dz 



1. JS^ifl 



■log<^ (2^e)■ 



1 



(c) 1 

< - 

~ 2 



log|(2^c) [QSy,.'Q 
log{(27re) [QS-i'Q]^::'} 



f G") 

Step (a) follows from the fact that q^i, and q-^l yield the 

f G) 

same moments of the quadratic form logg^^ . Step (b) is a 
well known formula on the determinant of matrix. Step (c) 



follows from Ej^i ^ S. Thus 



F,(S|0)< ilog{(27re) [QE^^Q]^^/} 

is concluded. Reverse inequality holds by letting Px^^x^i be 
Gaussian with covariance matrix S. ■ 



B. Proof of Lemma [7| 

In this appendix we prove Lemma [T] 

We write a unitary matrix Q as Q ^ [lij]^ where qij stands 
for the entry of Q. The unitary matrix Q transforms 
into QX'^. Set Q = Q^A and let be the (i, j) entry 

of QM. The following lemma states an important property on 
the distribution of Gaussian random vector . This lemma 
is a basis of the proof of Lemma [T] 

Lemma 10: For any i = 1, 2, • • • , ii', we have the following. 

1 . L . 

Z, = - Y.y,,Z, + —Y,^Y,+N,, (76) 



9ii .-^ '^N. 
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where 



a 



(77) 



Vij, j £ {1, 2, • • ■ , K} — {i} are suitable constants and Ni is a 

zero mean Gaussian random variables with variance — . For 

ga 

each i ^ S, Ni is independent of Zj,j G {1, 2, • ■ • , K} — {i} 
and YjJ 6 A. 

Proof: Without loss of generality we may assume i = 1. 
Since = AX^ + N^, we have 



Since Z^' = QX^, we have 



The density function pzKyi-iz^ ^y'") of is given 

by 



1 



'2 J^;.Jfv-C- 



(27re)^ |SzA-y^| 



where E^^^^ has the following form: 



T 1 



"TV-!; 



Set 



2 ' 



A 



(78) 



Now, we consider the following partition of 1]^]^^^ 



511 


'512" 


512 


G22 



where gn, 512, and G22 are scalar, K + L — 1 dimensional 
vector, and {K + L — 1) x {K + L — 1) matrix, respectively. It 
is obvious from the above partition of T,~]fYL that we have 
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.912 



fc=l '^N^ 



[vi2 ■ ■ -i^ikPiiPu ■■■Pii 



It is well known that S 



-1 

ZKyL 



has the following expression: 



.911 


*.9l2 


512 


G22 



" 1 


'0121 




'.911 


'O12 


.-^312 






O12 


G22 





Set 



A 



111 

Then, we have 



911 



3l2 



zi + 

511 



512- (80) 



[z y ]Y.zKYL 













511 


*5l2' 








.512 


G22 


















"511 




t 


012 


O12 


G22 ' 


- ^512*512 



ni 



11] 



(81) 



From (|78]l-(l80ll, we have 



n-i = Zl 



= Zl 



1 ^ 
511 t< 



3=2 



L y 



Zj + ^J2l3ijyj 



1 ^ 

— E-i.-. 

^ j=2 



(82) 



It can be seen from dsTl) and (|82] | that the random variable Ni 
defined by 



Ni^Zi 



is a zero mean Gaussian random variable with variance — 

911 

and is independent of and Y . This completes the proof 
of Lemma [TO] ■ 
The followings are two variants of the entropy power 
inequality. 

Lemma 11: Let Ui,i = 1,2,3 be n dimensional random 
vectors with densities and let T be a random variable taking 
values in a finite set. We assume that U3 is independent of 
Ui, U2, and T. Then, we have 

J_ .2-/i(i72+i73|f/iT) > J_QT:h{U2\UiT) , 1 „ffe(C/,3) 
27rc — 27rc 27rc 

Lemma 12: Let fj^, i = 1,2,3 be n random vectors with 
densities. Let Ti,T2 be random variables taking values in 
finite sets. We assume that those five random variables form 
a Markov chain (Ti,t/i) U3 ^ (72, {72) in this order. 
Then, we have 



2ire'- 



,^h(Ul+U2\U3TiT2) 



— 27rc 27rc 



(79) Proof of Lemma ^ By Lemma [Tol we have 



■9" '^^/^ 



AT, 



(83) 



1 


1 t„ ■ 

—1 912 


.012 


_ 



where Ni is a vector of n independent copies of zero mean 
Gaussian random variables with variance — . For each i G 

9ii 

A, Ni is independent of Zj^j G {1,2,---, A'} — {i] and 
l^j , j G A. Set 

A 1 



-h{z^\zLw 
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Furthermore, for fc G A, define 



Using ( |90l ) iteratively for j ~ 1, 2, • • • , s — 1, we have 



j=k ^N, 



Applying Lemma [TT| to ( |83] |. we have 



£!^1> 1 1 + (84) 

27re (gji)^27rc .g^i 

On the quantity h{^i\z^y W^) in the right member of ( |84] |. 
we have the following chain of equalities: 

n/i(") - /i(Z,|*i, Z^) + W^) 
^ W")-^log [27re(.g,,;)-i] +/i(*i|X^,M^^).(85) 

Step (a) follows from that Z^ can be obtained from 

by the invertible matrix Q. Step (b) follows from the Markov 

chain 



Z, -^{^i,Z 
From ([85), we have 



27re 
Substituting 



mto 



27Te ^"'27Te'' 
we obtain 



27re 



> 



1 



1 



27re 27re 



Solving ( [87] l with respect to 

„2/i<") 



we obtain 



27re 



> 



1 

27re' 



(86) 



(87) 



(88) 



Next, we evaluate a lower bound of ei''-^'^^^-^'^ ■^'^ K Note 
that for j = 1, 2, • • • , s — 1 we have the following Markov 
chain: 



K 



W„^Y, I . (89) 



Based on dgUl, we apply Lemma [12] to -i-e*''(*:' l^"^'^"") 



for j = 1, 2, • • • , s — 1. Then, for j = 1, 2, 
the following chains of inequalities : 
1 



, s — 1, we have 



27re 



27re 



27re 



27re 



27re 



-2r^ 



(90) 



27re 



-2r': 



y ^2 



(91) 



Combining (|72ll, ((MJ. and (|9T]), we have 



^2h* 



27rc 



j2 



completing the proof. 
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